Information processing apparatus

ABSTRACT

An information processing apparatus includes a first label determination unit that determines a first label from information included in an e-mail, a second label determination unit that determines a second label from a result of a response made to the e-mail by a user, and a third label determination unit that determines a third label as a negative example for machine learning which is imparted to the e-mail, in a case where the first label and the second label do not correspond to each other.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2017-184582 filed Sep. 26, 2017.

BACKGROUND Technical Field

The present invention relates to an information processing apparatus.

SUMMARY

According to an aspect of the invention, there is provided aninformation processing apparatus including a first label determinationunit that determines a first label from information included in ane-mail, a second label determination unit that determines a second labelfrom a result of a response made to the e-mail by a user, and a thirdlabel determination unit that determines a third label as a negativeexample for machine learning which is imparted to the e-mail, in a casewhere the first label and the second label do not correspond to eachother.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiment(s) of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a conceptual module configuration diagram illustrating aconfiguration example according to this exemplary embodiment;

FIG. 2 is a diagram illustrating a system configuration example usingthis exemplary embodiment;

FIG. 3 is a flowchart illustrating a processing example according tothis exemplary embodiment;

FIG. 4 is a diagram illustrating an example of a data structure of amail information table;

FIG. 5 is a diagram illustrating an example of a data structure of arule table with meta information label;

FIG. 6 is a diagram illustrating an example of a data structure of anaction history information table;

FIG. 7 is a diagram illustrating an example of a data structure of arule table with action history label;

FIG. 8 is a diagram illustrating an example of a data structure of alearning data table;

FIG. 9 is a diagram illustrating an example of a data structure of alearning data table;

FIG. 10 is a diagram illustrating an example of a data structure of alearning data table;

FIG. 11 is a diagram illustrating an example of a data structure of alearning data table; and

FIG. 12 is a block diagram illustrating a hardware configuration exampleof a computer for realizing this exemplary embodiment.

DETAILED DESCRIPTION

Hereinafter, an example of an exemplary embodiment in realizing theinvention will be described with reference to the accompanying drawings.

FIG. 1 is a conceptual module configuration diagram illustrating aconfiguration example of this exemplary embodiment.

Meanwhile, the term “module” refers to components such as software(computer programs) and hardware which are typically capable of beinglogically separated. Consequently, the term “module” in this exemplaryembodiment not only refers to modules in a computer program, but also tomodules in a hardware configuration. Thus, this exemplary embodimentalso serves as a description of a computer program (a program thatcauses a computer to execute respective operations, a program thatcauses a computer to function as respective units, or a program thatcauses a computer to realize respective functions), a system, and amethod for inducing functionality as such modules. Meanwhile, althoughterms like “store” and “record” and their equivalents may be used in thedescription for the sake of convenience, these terms mean that a storageapparatus is made to store information or that control is applied tocause a storage apparatus to store information in the case where theexemplary embodiment is a computer program. In addition, while modulesmay be made to correspond with function on a one-to-one basis, someimplementations may be configured such that one program constitutes onemodule, such that one program constitutes multiple modules, orconversely, such that multiple programs constitute one module. Moreover,plural modules may be executed by one computer, but one module may alsobe executed by plural computers in a distributed or parallel computingenvironment. Meanwhile, a single module may also contain other modules.In addition, the term “connection” may be used hereinafter to denotelogical connections (such as the transmission and reception of data,instructions, a referential relationship between pieces of data, andlog-in) in addition to physical connections. The term “predetermined”refers to something being determined prior to the processing inquestion, and obviously denotes something that is determined before aprocess according to the exemplary embodiment starts, but may alsodenote something that is determined after a process according to theexemplary embodiment has started but before the processing in question,according to conditions or states at that time, or according toconditions or states up to that time. In the case of plural“predetermined values”, the predetermined values may be respectivelydifferent values, or two or more values (this obviously also includesthe case of all values) which are the same. Additionally, statements tothe effect of “B is conducted in the case of A” are used to denote thata determination is made regarding whether or not A holds true, and B isconducted in the case where it is determined that A holds true. However,this excludes cases where the determination of whether or not A holdstrue may be omitted. Additionally, the case of the listing of thingssuch as “A, B, C” is illustrative listing unless otherwise indicated,and includes a case where only one of them is selected (for example,only A).

In addition, the terms “system” and “apparatus” not only encompassconfigurations in which plural computers, hardware, or apparatus areconnected by a communication medium such as a network (includingconnections that support 1-to-1 communication), but also encompassconfigurations realized by a single computer, hardware, or apparatus.The terms “apparatus” and “system” are used interchangeably. Obviously,the term “system” does not include merely artificially arranged socialconstructs (social systems).

Also, every time a process is conducted by each module or every timeplural processes are conducted within a module, information to beprocessed is retrieved from a storage apparatus, and the processingresults are written back to the storage apparatus after the processing.Consequently, description of the retrieval from a storage apparatusbefore processing and the writing back to a storage apparatus afterprocessing may be omitted in some cases. Meanwhile, the storageapparatus herein may include hard disks, random access memory (RAM), anexternal storage medium, storage apparatus accessed via a communicationlink, and registers, and the like inside a central processing unit(CPU).

An information processing apparatus 100 which is this exemplaryembodiment generates teacher data for machine learning using an e-mail,and includes a meta information processing module 105, an action historyprocessing module 130, a label determination module 155, and a learningdata database 160 as illustrated in the example of FIG. 1. For example,the information processing apparatus 100 generates teacher data with acase name with the subject of an e-mail and an action as a clue.

E-mail is a tool for two-way communication, and includes a transmitter'sintention and a recipient's intention. Here, the transmitter's intentionis reflected on contents of the e-mail, but the recipient's intention isoften reflected on an action occurring by the received e-mail. In thiscase, a label having no recipient's intention reflected thereto may beattached in a case where a label is imparted using only the contents ofthe e-mail.

In addition, the transmitter's intention and the recipient's intentiondo not necessarily match each other. In a case where the transmitter'sintention and the recipient's intention are different from each other,the accuracy of machine learning may be further increased in a casewhere the teacher data is treated as having a meaning of a negativeexample.

The meta information processing module 105 includes a mail database 110,a mail acquisition module 115, a rule-with-meta-information-labelstorage module 120, and a meta information label determination module125. The meta information processing module 105 performs a labelimpartation process based on meta information. The meta information asmentioned herein is generated from information included in the e-mail,and may be, for example, a predetermined character string extracted fromthe subject (title) of the e-mail.

The mail database 110 is connected to the mail acquisition module 115.The mail database 110 stores the target mail.

The mail acquisition module 115 is connected to the mail database 110and the meta information label determination module 125. The mailacquisition module 115 acquires information (for example, e-mailidentification information (e-mail Identification (ID)), the title, thebody, address information (TO), transmission source information (FROM),and the like) on the e-mail from the mail database 110, and transmitsthe acquired information to the meta information label determinationmodule 125.

The rule-with-meta-information-label storage module 120 is connected tothe meta information label determination module 125. Therule-with-meta-information-label storage module 120 stores labels andrules with label (for example, a keyword, regular expression, and thelike).

The meta information label determination module 125 is connected to themail acquisition module 115, the rule-with-meta-information-labelstorage module 120, and the label determination module 155. The metainformation label determination module 125 determines a first label(hereinafter, also referred to as a meta information label) frominformation included in the e-mail transmitted from the mail acquisitionmodule 115.

The “information included in the e-mail” may include information whichis required to transmit and receive the e-mail, in addition to the titleand the body of the e-mail (information readable by a person). Examplesof the information required to transmit and receive the e-mail includean e-mail ID, address information, transmission source information, andthe like.

As a first label determination method, specifically, the first label isdetermined in accordance with a first rule included in therule-with-meta-information-label storage module 120 in which the “firstlabel” and a “rule for determining the first label are associated witheach other with the information included in the e-mail as a target”.

The action history processing module 130 includes an action historydatabase 135, an action history acquisition module 140, arule-with-action-history-label storage module 145, and an action historylabel determination module 150. The action history processing module 130performs a label impartation process based on the history of actionsperformed by a user, with respect to the e-mail.

The action history database 135 is connected to the action historyacquisition module 140. The action history database 135 stores thehistory of the user's actions performed on the e-mail.

The action history acquisition module 140 is connected to the actionhistory database 135 and the action history label determination module150. The action history acquisition module 140 acquires the history ofactions (for example, reply, forwarding, and the like) from the actionhistory database 135 by using an e-mail ID as a clue, and transmits theacquired history to the action history label determination module 150.

The rule-with-action-history-label storage module 145 is connected tothe action history label determination module 150. Therule-with-action-history-label storage module 145 stores labels andrules with label (for example, the name of an action, and the like).

The action history label determination module 150 is connected to theaction history acquisition module 140, therule-with-action-history-label storage module 145, and the labeldetermination module 155. The action history label determination module150 determines a second label (hereinafter, also referred to as anaction history label) from the history of the e-mail which istransmitted from the action history acquisition module 140. Naturally, atarget e-mail of the action history label determination module 150 and atarget e-mail of the meta information label determination module 125 arethe same as each other (that is, the e-mails have the same e-mail ID).

The “history of an e-mail” includes the history of operations performedby a user (an operator, a recipient, or the like) by using informationincluded in the e-mail, in addition to the history of the e-mail itself.Examples of the history of operations include an operation of copyingthe information included in the e-mail to another application (forexample, a schedule management application or the like), in addition tooperations such as reply and forwarding of the e-mail.

As a second label determination method, specifically, the second labelis determined in accordance with a second rule included in therule-with-action-history-label storage module 145 in which the “secondlabel” and a “rule for determining the second label are associated witheach other with the history of the e-mail as a target”.

For example, the action history label determination module 150determines the second label from results of the user's response to thee-mail.

The label determination module 155 is connected to the meta informationlabel determination module 125 of the meta information processing module105, the action history label determination module 150 of the actionhistory processing module 130, and the learning data database 160.

The label determination module 155 determines a third label which is afinal label, on the basis of the first label determined by the metainformation label determination module 125 and the second labeldetermined by the action history label determination module 150.

For example, the label determination module 155 determines the thirdlabel as a negative example for machine learning which is to be impartedto the e-mail in a case where the first label determined by the metainformation label determination module 125 and the second labeldetermined by the action history label determination module 150 do notcorrespond to each other.

The label determination module 155 may adopt either the first label orthe second label as the third label in a case where the first label andthe second label do not correspond to each other.

In a case where either the first label or the second label is set to beno label, the label determination module 155 may adopt either the firstlabel or the second label which is not set to be no label as the thirdlabel.

In addition, the label determination module 155 determines a third labelas a positive example for machine learning which is to be imparted tothe e-mail, in a case where the first label determined by the metainformation label determination module 125 and the second labeldetermined by the action history label determination module 150correspond to each other.

The label determination module 155 may adopt the first label or thesecond label as the third label, in a case where the first label and thesecond label correspond to each other. Here, the “case where the firstlabel and the second label correspond to each other” corresponds to acase where the first label and the second label are the same as eachother.

In a case where both the first label determined by the meta informationlabel determination module 125 and the second label determined by theaction history label determination module 150 are set to be no label,the label determination module 155 may determine no label for machinelearning which is to be imparted to the e-mail. In a case where eitherthe first label or the second label is set to be no label, it ispossible to reduce determination to be no label, as compared to a casewhere no label for machine learning which is to be imparted to thee-mail is determined.

In addition, the label determination module 155 may determine no labelfor machine learning which is to be imparted to the e-mail in a casewhere either the first label determined by the meta information labeldetermination module 125 or the second label determined by the actionhistory label determination module 150 is set to be no label. In a casewhere both the first label and the second label are set to be no label,it is possible to reduce the impartation of a label with low accuracy,as compared to a case where no label for machine learning which is to beimparted to the e-mail is determined.

In addition, the label determination module 155 may perform thefollowing processing.

The label determination module 155 determines a third label for machinelearning which is to be imparted to the e-mail from the first labeldetermined by the meta information label determination module 125 andthe second label determined by the action history label determinationmodule 150.

As a third label determination method, specifically, a label isdetermined in accordance with a third rule in which a set of the “firstlabel” and the “second label” is associated with the “third label”.Information indicating whether the third label is a positive example ora negative example as teacher data may be added.

In addition, the label determination module 155 may adopt the firstlabel or the second label as the third label in a case where the firstlabel and the second label are the same as each other.

The label determination module 155 may set the third label to be nolabel in a case where both the first label and the second label are setto be no label. The case of setting to be “no label” does not correspondto the first rule included in the rule-with-meta-information-labelstorage module 120 and the second rule included in therule-with-action-history-label storage module 145.

In addition, the label determination module 155 may adopt either thefirst label or the second label as the third label in a case where thefirst label and the second label are different from each other.

In addition, the label determination module 155 may adopt the secondlabel as the third label in a case where the first label and the secondlabel are different from each other. This is a determination attachingmuch importance to a recipient's action.

In a case where either the first label or the second label is set to beno label (a case where one label is set to be no label and the otherlabel is not set to be no label (having any one label)), the labeldetermination module 155 may adopt either the first label or the secondlabel (the other label having any one label) which is not set to be nolabel as the third label.

In addition, the label determination module 155 may determine whetherthe third label is a positive example or a negative example as teacherinformation.

In a case where both the first label and the second label are the sameas each other, the label determination module 155 may determine that thethird label is a positive example.

In a case where the first label and the second label are different fromeach other, the label determination module 155 may determine that thethird label is a negative example.

As a determination process in the label determination module 155, thecase where the first label and the second label are different from eachother may include a case where any one of the first label and the secondlabel is set to be no label.

In a case where both the first label and the second label are not set tobe no label, the label determination module 155 may determine that thethird label is not a positive example. The “being not a positiveexample” as mentioned herein may refer setting to be no label or settingto be a negative example in a target label. Meanwhile, the “targetlabel” as mentioned herein is a predetermined label, and the label maynot be extracted by the meta information label determination module 125and the action history label determination module 150 and is thusextracted as a label of a negative example. For example, although thetarget label is “important”, “important” is not included in the title,and actions such as reply and forwarding are not also performed.Accordingly, “no label” is set in the meta information labeldetermination module 125 and the action history label determinationmodule 150. However, the label determination module 155 sets the e-mailas a negative example of “important” and registers the e-mail in thelearning data database 160 as teacher data.

The learning data database 160 is connected to the label determinationmodule 155. The learning data database 160 stores teacher data in whichthe third label determined by the label determination module 155 and thee-mail are associated with each other.

FIG. 2 is a diagram illustrating a system configuration example usingthis exemplary embodiment.

The information processing apparatus 100, a user terminal 210A, a userterminal 210B, a user terminal 210C, a mail processing apparatus 220,and a machine learning apparatus 230 are connected to each other througha communication line 290. The communication line 290 may be a wirelessline, a wired line, or a combination thereof, and may be, for example,the Internet, an intranet, or the like as communication infrastructure.In addition, the functions of the information processing apparatus 100,the mail processing apparatus 220, and the machine learning apparatus230 may be realized as cloud service.

The mail processing apparatus 220 performs processing related to theforwarding of an e-mail. Examples of the mail processing apparatusinclude a POP server, an SMTP server, a WEB mail server, and the like.In a case where the mail processing apparatus 220 collects e-mails, themail database 110 may be provided within the mail processing apparatus220 instead of the information processing apparatus 100.

The mail processing apparatus 220 or the information processingapparatus 100 collects histories of processing performed on an e-mailfrom the mail processing apparatus 220 and the user terminals 210, andstores the collected histories in the action history database 135.

The machine learning apparatus 230 performs machine learning by usingteacher data stored in the learning data database 160 of the informationprocessing apparatus 100. The learning data database 160 may be providedwithin the machine learning apparatus 230 instead of the informationprocessing apparatus 100. As the machine learning, identificationdevices such as a neural network, a decision tree, and a support vectormachine may be used.

The information processing apparatus 100 performs, for example, thefollowing processing. In particular, this description (description inthis paragraph number) is given to facilitate the understanding of thisexemplary embodiment, and limited analysis using this description is notintended. It is natural that determination should not be performedregarding whether the invention for which a patent is sought isdescribed in the detailed description of the invention (Patent ActArticle 36(6)(i)) by using this description part.

In general, high costs are required to create teacher data. For example,in a case where the teacher data is created by hand, costs areincreased. In a method in which the teacher data is automaticallycreated using an entry word, the entry word does not necessarilyrepresent contents of the body text, and the quality of annotation isinferior. For example, in two-way communication such as an e-mail, asubject (entry word) attached by a writer (transmitter) and a subjectrecognized by a receiver (recipient) may be different from each other.Specifically, the entirety of an e-mail written to be [important] is notnecessarily important to the recipient.

The information processing apparatus 100 improves the quality of theteacher data while suppressing creation costs of the teacher data.

The information processing apparatus 100 improves the quality of theteacher data in consideration of an action on an e-mail (for example,reply, forwarding, the impartation of task, sharing, the registration ofa schedule, and the like) and meta information (the number ofrecipients, the presence or absence of an attached file, and the like)which are added.

For example, in a case where a keyword (expression) related to a labelis included in the subject of an e-mail, there is a strong possibilitythat the e-mail is a positive example. Specifically, in a case where thesubject is “[important] ◯◯”, “[reply required] ΔΔ”, or the like, theformer subject is highly likely to indicate an e-mail of “important”,and the latter subject is highly likely to indicate an e-mail of “replyrequired”.

There is a higher possibility that an e-mail having been subjected to anaction related to a label is a positive example. For example, there is astrong possibility that “reply” is performed as an action on an e-mailin a case where the e-mail is an e-mail of “reply required” (an e-mailto which “reply required” as a third label is imparted), “reply”,“forwarding”, and “impartation of task” are performed as actions on ane-mail in a case where the e-mail is an e-mail of “important” (an e-mailto which “important” as a third label is imparted), and “reply” and“registration of calendar” are performed as actions on an e-mail in acase where the e-mail is an e-mail of “adjustment of schedule” (ane-mail to which “adjustment of schedule” as a third label is imparted).

Consequently, the information processing apparatus 100 determines afirst label from information included in an e-mail, determines a secondlabel from an action on the e-mail, and determines a label (third label)to be imparted to the e-mail by using both the first label and thesecond label.

FIG. 3 is a flowchart illustrating a processing example according tothis exemplary embodiment.

In step S302, the mail acquisition module 115 acquires an e-mail fromthe mail database 110. For example, the mail information table 400 isstored in the mail database 110, and any one row (e-mail) is selected.

FIG. 4 is a diagram illustrating an example of a data structure of themail information table 400. The mail information table 400 includes amail ID column 410, a title column 420, a transmitter column 430, arecipient column 440, and an attached file column 450. The mail IDcolumn 410 stores information (mail ID) for uniquely identifying ane-mail in this exemplary embodiment. The title column 420 stores thetitle of the e-mail. The transmitter column 430 stores the transmitterof the e-mail. The recipient column 440 stores the recipient of thee-mail. The attached file column 450 stores an attached file of thee-mail.

For example, in FIG. 4, an e-mail of mail 1 includes title: “[important]change of specification of ◯◯”, transmitter: “user A”, recipient: “usersB, C, and D”, and attached file: “specifications.pdf”, an e-mail of mail2 includes title: “[important] seminar information”, transmitter: “userA”, recipient: “users B, C, and D”, and attached file: “seminardata.pptx”, an e-mail of mail 3 includes title: “3/21 weekly report”,transmitter: “user A”, recipient: “users D, E, and F”, and attachedfile: “none”.

In step S304, the meta information label determination module 125determines a meta information label (first label) of a target e-mail inaccordance with the rule-with-meta-information-label storage module 120.For example, a rule table with meta information label 500 included inthe rule-with-meta-information-label storage module 120 is used.

FIG. 5 is a diagram illustrating an example of a data structure of therule table with meta information label 500. The rule table with metainformation label 500 includes a rule ID column 510, a label column 520,and a rule column 530. The rule ID column 510 stores information (ruleID) for uniquely identifying a rule in this exemplary embodiment. Thelabel column 520 stores a label in a case where the rule is suitable.The rule column 530 stores a rule.

For example, in FIG. 5, rule 1 is conditioned to be suitable for “titleincludes “[important]” and “[urgent]”, and attached file is present” indetermining a first label: “important”. Rule 2 is conditioned to besuitable for “title includes “[fixed]” and expression of date” indetermining a first label: “adjustment of schedule”. Rule 3 isconditioned to be suitable for “title includes “[reply required]” or“[reply desired]”” in determining a first label: “reply required”.

In a case where the rule table with meta information label 500illustrated in the example of FIG. 5 is applied to the mail informationtable 400 illustrated in the example of FIG. 4, a first label isdetermined as follows.

Mail 1: an “important” label is imparted because of being suitable forthe rule 1.

Mail 2: an “important” label is imparted because of being suitable forthe rule 1.

Mail 3: “no label” is imparted because of being unsuitable for both therules 1 and 2.

In step S306, the action history acquisition module 140 acquires thehistory of actions related to an e-mail from the action history database135. For example, an action history information table 600 is stored inthe action history database 135, and the row (action on the e-mail) ofthe e-mail which is acquired in step S302 is selected.

FIG. 6 is a diagram illustrating an example of a data structure of theaction history information table 600. The action history informationtable 600 includes a mail ID column 610, an operator column 620, and anaction column 630. The mail ID column 610 stores a mail ID. The operatorcolumn 620 stores an operator (in general, a recipient) for the mail.The action column 630 stores an action on the mail.

For example, in FIG. 6, an action: “reply” is performed by an operator:“user B” with respect to the mail 1, an action: “forwarding” isperformed by an operator: “user C” with respect to the mail 1, anaction: “reply” is performed by an operator: “user D” with respect tothe mail 2, and an action: “reply” is performed by an operator: “user F”with respect to the mail 3.

In step S308, the action history label determination module 150determines an action history label (second label) of a target e-mail inaccordance with the rule-with-action-history-label storage module 145.For example, a rule table with action history label 700 included in therule-with-action-history-label storage module 145 is used.

FIG. 7 is a diagram illustrating an example of a data structure of therule table with action history label 700. The rule table with actionhistory label 700 includes a rule ID column 710, a label column 720, anda rule column 730. The rule ID column 710 stores a rule ID. The labelcolumn 720 stores a label in a case where the rule is suitable. The rulecolumn 730 stores a rule.

For example, in FIG. 7, rule A1 is conditioned to be suitable for “thenumber of recipients having performed reply and forwarding is equal toor greater than half of recipients” in determining a second label:“important”. Rule A2 is conditioned to be suitable for “recipient isregistered in calendar” in determining a second label: “adjustment ofschedule”. In addition, for example, a rule conditioned to be suitablefor “the number of recipients having performed reply is equal to orgreater than half of recipients” in determining a second label: “replyrequired” may be added.

In a case where the rule table with action history label 700 illustratedin the example of FIG. 7 is applied to the action history informationtable 600 illustrated in the example of FIG. 6, a second label isdetermined as follows.

Mail 1: an “important” label is imparted because of being suitable forthe rule A1.

Mail 2: “no label” is imparted because of being unsuitable for both therules A1 and A2.

Mail 3: “no label” is imparted because of being unsuitable for both therules A1 and A2.

In step S310, the label determination module 155 determines a label(third label) of a target e-mail. Hereinafter, four examples (X1, X2,X3, and X4) are shown.

(X1) For example, (1) in a case where the label based on metainformation and the label based on the history of actions are the sameas each other, the label based on meta information or the label based onthe history of actions (since both the labels are the same as eachother, either one may be adopted) is adopted as a final label, (2) in acase where both the label based on meta information and the label basedon the history of actions indicate no label, a final label is set to beno label, and (3) in a case where either the label based on metainformation or the label based on the history of actions indicates nolabel, a learning data table 800 is generated as a processing result ina case where a rule for adopting either the label based on metainformation or the label based on the history of actions, which does notindicate no label as a final label, is applied.

FIG. 8 is a diagram illustrating an example of a data structure of thelearning data table 800. The learning data table 800 includes a mail IDcolumn 810, a label-based-on-meta-information column 820, alabel-based-on-history-of-actions column 830, and a final label column840. The mail ID column 810 a mail ID. Thelabel-based-on-meta-information column 820 stores a label (first label)based on meta information. The label-based-on-history-of-actions column830 stores a label (second label) based on the history of actions. Thefinal label column 840 stores a final label (third label).

For example, mail 1 includes label: “important” based on metainformation, label: “important” based on the history of actions, andfinal label: “important”, mail 2 includes label: “important” based onmeta information, label: “no label” based on the history of actions, andfinal label: “important”, and mail 3 includes label: “no label” based onmeta information, label: “no label” based on the history of actions, andfinal label: “no label”.

That is, the mails are as follows.

Mail 1: an “important” label is imparted because the “important” labelis imparted for all of the labels.

Mail 2: an “important” label is imparted because the label based on metainformation is set to be “important”.

Mail 3: “no label” (not used for learning data) is imparted because the“no label” is imparted for all of the labels.

(X2) In addition, for example, (1) in a case where the label based onmeta information and the label based on the history of actions are thesame as each other, the label based on meta information or the labelbased on the history of actions (since both the labels are the same aseach other, either one may be adopted) is adopted as a final label, (2)in a case where both the label based on meta information and the labelbased on the history of actions indicate no label, a final label is setto be no label, and (3) in a case where both the label based on metainformation and the label based on the history of actions are differentfrom each other, a learning data table 900 is generated as a processingresult in a case where a rule for adopting the label based on thehistory of actions is adopted as a final label.

FIG. 9 is a diagram illustrating an example of a data structure of thelearning data table 900.

The learning data table 900 includes a mail ID column 910, alabel-based-on-meta-information column 920, alabel-based-on-history-of-actions column 930, and a final label column940. The mail ID column 910 stores a rule ID. Thelabel-based-on-meta-information column 920 stores a label (first label)based on meta information. The label-based-on-history-of-actions column930 stores a label (second label) based on the history of actions. Thefinal label column 940 stores a final label (third label).

For example, the mail 1 includes label: “important” based on metainformation, label: “important” based on the history of actions, andfinal label: “important”, the mail 2 includes label: “important” basedon meta information, label: “no label” based on the history of actions,and final label: “no label”, and the mail 3 includes label: “no label”based on meta information, label: “no label” based on the history ofactions, and final label: “important”.

That is, the mails are as follows.

Mail 1: an “important” label is imparted because the “important” labelis imparted for all of the labels.

Mail 2: “no label” (not used for learning data) is imparted because allof the labels are different from each other and the label based on thehistory of actions is set to be “no label”.

Mail 3: “no label” (not used for learning data) is imparted because the“no label” is imparted for all of the labels.

(X3) In addition, a case including whether a final label is a positiveexample or a negative example as teacher information may be as follows.

For example, (1) in a case where the label based on meta information andthe label based on the history of actions are the same as each other,the label based on meta information or the label based on the history ofactions (since both the labels are the same as each other, either onemay be adopted) is adopted as a final label, (2) in a case where boththe label based on meta information and the label based on the historyof actions indicate no label, a final label is set to be no label, (3)in a case where either the label based on meta information or the labelbased on the history of actions indicates no label, either the labelbased on meta information or the label based on the history of actions,which does not indicate no label as a final label, is applied, (4) in acase where both the label based on meta information and the label basedon the history of actions are the same as each other, a final label isset to be a positive example, (5) in a case where the label based onmeta information and the label based on the history of actions aredifferent from each other (one of the labels includes no label), alearning data table 1000 is generated as a processing result in a casewhere a rule for adopting a negative example as a final label isapplied.

FIG. 10 is a diagram illustrating an example of a data structure of thelearning data table 1000. The learning data table 1000 includes a mailID column 1010, a label-based-on-meta-information column 1020, alabel-based-on-history-of-actions column 1030, a final label column1040, and a positive example/negative example column 1050. The mail IDcolumn 1010 stores a rule ID. The label-based-on-meta-information column1020 stores a label (first label) based on meta information. Thelabel-based-on-history-of-actions column 1030 stores a label (secondlabel) based on the history of actions. The final label column 1040stores a final label (third label). The positive example/negativeexample column 1050 stores either a positive example or a negativeexample.

For example, mail 1 includes label: “important” based on metainformation, label: “important” based on the history of actions, finallabel: “important”, and positive example/negative example: “positiveexample”, mail 2 includes label: “important” based on meta information,label: “no label” based on the history of actions, final label:“important”, and positive example/negative example: “negative example”,and mail 3 includes label: “no label” based on meta information, label:“no label” based on the history of actions, final label: “no label”, andpositive example/negative example: “- (null)”.

That is, the mails are as follows.

Mail 1: an “important” label which is a positive example is impartedbecause the “important” label is imparted for all of the labels.

Mail 2: “important” which is a negative example is imparted because alabel based on meta information is set to be “important” and a labelbased on the history of actions is set to be “no label”.

Mail 3: “no label” (not used for learning data) is imparted because the“no label” is imparted for all of the labels.

Since it is easy to achieve a high level of accuracy in a case wherelearning data includes a negative example (near-miss example) close to apositive example in machine learning, there is a strong possibility thatthe level of accuracy is increased more in a case where the mail 2 isset to be a negative example than in a case where the mail 3 is set tobe a negative example.

(X4) In addition, for example, (1) in a case where the label based onmeta information and the label based on the history of actions are thesame as each other, the label based on meta information or the labelbased on the history of actions (since both the labels are the same aseach other, either one may be adopted) is adopted as a final label, (2)in a case where both the label based on meta information and the labelbased on the history of actions indicate no label, a target label isadopted, and the label is set to be a negative example, (3) in a casewhere the label based on meta information and the label based on thehistory of actions are different from each other, a label based on thehistory of actions is adopted as a final label, and (4) in a case whereboth the label based on meta information and the label based on thehistory of actions are the same as each other, a learning data table1100 is generated as a processing result in a case where a rule forsetting the final label to be a positive example is applied. Meanwhile,the “target label” in the example of FIG. 11 is “important” which isdetermined in advance.

FIG. 11 is a diagram illustrating an example of a data structure of thelearning data table 1100. The learning data table 1100 includes a mailID column 1110, a label-based-on-meta-information column 1120, a labelcolumn 1130 based on the history of actions, a final label column 1140,and a positive example/negative example column 1150. The mail ID column1110 stores a rule ID. The label-based-on-meta-information column 1120stores a label (first label) based on meta information. The label column1130 based on the history of actions stores a label (second label) basedon the history of actions. The final label column 1140 stores a finallabel (third label). The positive example/negative example column 1150stores either a positive example or a negative example.

For example, mail 1 includes label: “important” based on metainformation, label: “important” based on the history of actions, finallabel: “important”, and positive example/negative example: “positiveexample”, mail 2 includes label: “important” based on meta information,label: “no label” based on the history of actions, final label: “nolabel”, and positive example/negative example: “- (null)”, and mail 3includes label: “no label” based on meta information, label: “no label”based on the history of actions, final label: “important”, and positiveexample/negative example: “negative example”.

That is, the mails are as follows.

Mail 1: an “important” label which is a positive example is impartedbecause the “important” label is imparted for all of the labels.

Mail 2: “no label” (not used for learning data) is imparted because allof the labels are different from each other and the label based on thehistory of actions is set to be “no label”.

Mail 3: “important” (negative example) which is a target label isimparted because “no label” is imparted for all of the labels.

In step S312, the label determination module 155 stores learning data inthe learning data database 160. Specifically, the learning data table800, the learning data table 900, the information processing apparatus100, and the learning data table 1100 which are illustrated in theexamples of FIGS. 8 to 11 are stored in the learning data database 160.Meanwhile, the learning data table 800 and the learning data table 900may include learning data constituted by only a mail ID column and afinal label column, and the learning data table 1000 and the learningdata table 1100 may include learning data constituted by only a mail IDcolumn, a final label column, and a positive example/negative examplecolumn.

In addition, regarding (1) a set of the processes of step S302 and stepS304 and (2) a set of the processes of step S306 and step S308, any setof the processes may be performed first, or the sets of the processesmay be performed in parallel.

Meanwhile, a hardware configuration of a computer executing a program asthis exemplary embodiment is a general computer as illustrated in FIG.12, and specifically, is a personal computer, a computer serving as aserver, or the like. That is, as a specific example, a CPU 1201 is usedas a processing unit (computational unit), and a RAM 1202, a ROM 1203,and an HD 1204 are used as storage devices. As the HD 1204, for example,a hard disk or a Solid State Drive (SSD) may be used. The computerincludes the CPU 1201 that executes programs such as the mailacquisition module 115, the meta information label determination module125, the action history acquisition module 140, the action history labeldetermination module 150, and the label determination module 155, theRAM 1202 that stores the programs and data, the ROM 1203 that storesprograms for starting up the computer, and the like, the HD 1204 whichis an auxiliary storage device (may be a flash memory or the like)including functions as the mail database 110, therule-with-meta-information-label storage module 120, the action historydatabase 135, the rule-with-action-history-label storage module 145, andthe learning data database 160, a reception device 1206 that receivesdata on the basis of a user's operation (including a motion, a sound, aneye gaze, and the like) with respect to a keyboard, a mouse, a touchscreen, a microphone, a camera (including an eye gaze detection camera,and the like), and the like, an output device 1205 such as a CRT, aliquid crystal display, or a speaker, a communication line interface1207, such as a network interface card, for connection to acommunication network, and a bus 1208 for transmitting and receivingdata by connecting the above-mentioned components. Plural computers maybe connected to each other through a network.

Regarding the exemplary embodiment based on a computer program among theabove-described exemplary embodiments, a system having this hardwareconfiguration is caused to read a computer program which is software,and the above-described exemplary embodiment is realized by thecooperation of software and hardware resources.

Meanwhile, a hardware configuration illustrated in FIG. 12 shows oneconfiguration example. This exemplary embodiment is not limited to theconfiguration illustrated in FIG. 12, and the information processingapparatus may be configured such that the modules described in thisexemplary embodiment are capable of being executed. For example, somemodules may be constituted by dedicated hardware (for example, anApplication Specific Integrated Circuit (ASIC) or the like), somemodules may be configured to be provided in an external system andconnected to each other through a communication line, or plural systemseach of which is illustrated in FIG. 12 may be connected to each otherthrough a communication line and operated in cooperation with eachother. In addition, the information processing apparatus may beparticularly incorporated into portable information communicationequipment (including a mobile phone, a smart phone, a mobile equipment,a wearable computer, and the like), an information appliance, a robot, acopying machine, a facsimile, a scanner, a printer, a multifunctionmachine (an image processing apparatus including any two or more of ascanner, a printer, a copying machine, and a facsimile), and the like,in addition to a personal computer.

Meanwhile, the programs described above may be provided through arecording medium which stores the programs, or may be provided through acommunication unit. In these cases, for example, the programs describedabove may be interpreted as an invention of “a computer-readablerecording medium that stores programs”.

The “computer-readable recording medium that stores programs” refers toa computer-readable recording medium that stores programs and is usedfor the installation and execution of the programs and the distributionof the programs.

Meanwhile, examples of the recording medium include a digital versatiledisk (DVD) having a format of “DVD-R, DVD-RW, DVD-RAM, or the like”which is a standard developed by the DVD forum or having a format of“DVD+R, DVD+RW, or the like” which is a standard developed by the DVD+RWalliance, a compact disk (CD) having a format of CD read only memory(CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), or the like, aBlu-ray Disc (registered trademark), a magneto-optical disk (MO), aflexible disk (FD), a magnetic tape, a hard disk, a read only memory(ROM), an electrically erasable programmable ROM (EEPROM (registeredtrademark)), a flash memory, a random access memory (RAM), a securedigital (SD) memory card, and the like.

The above-described programs or some of them may be stored anddistributed by recording on the recording medium. In addition, theprograms may be transmitted through communication, for example, by usinga transmission media of, for example, a wired network which is used fora local area network (LAN), a metropolitan area network (MAN), a widearea network (WAN), the Internet, an intranet, an extranet, and thelike, a wireless communication network, or a combination of these. Theprograms may be carried on carrier waves.

Further, the above-described programs may be a portion or all of otherprograms, or may be recorded on a recording medium along with otherprograms. The programs may be recorded on plural recording media bydividing the programs. The programs may be recorded in any format, suchas compression or encryption, as long as it is possible to restore theprograms.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing apparatus comprising: aprocessor configured to: determine a first label from informationincluded in an e-mail; determine a second label from a result of aresponse made to the e-mail by a user; and determine a third label as anegative example for machine learning which is imparted to the e-mail,in a case where the first label and the second label do not correspondto each other.
 2. The information processing apparatus according toclaim 1, wherein the processor is configured to adopt either the firstlabel or the second label as the third label in a case where the firstlabel and the second label do not correspond to each other.
 3. Theinformation processing apparatus according to claim 2, wherein theprocessor is configured to adopt either the first label or the secondlabel which is not set to be no label as the third label in a case whereeither the first label or the second label is set to be no label.
 4. Aninformation processing apparatus comprising: a processor configured to:determine a first label from information included in an e-mail;determine a second label from a result of a response made to the e-mailby a user; and determine a third label as a positive example for machinelearning which is imparted to the e-mail, in a case where the firstlabel and the second label correspond to each other.
 5. The informationprocessing apparatus according to claim 4, wherein the processor isconfigured to adopt the first label or the second label as the thirdlabel in a case where the first label and the second label correspond toeach other.